Generally, each frame of a video image is represented by a stream of pixel data corresponding to image values (analog, or digital with 1, 8, 16 or 24 bits per pixel) collected at defined positions along a plurality of horizontal scan lines. For example, a typical VGA computer image is represented by 640 8-bit pixels for each of 480 horizontal scan lines. There are two different ways in which the pixel data for a single video frame can be transmitted and displayed.
Referring to FIG. 1A, in a progressive scan display, such as a computer monitor, each line of a frame is displayed in order and the entire frame is displayed in a single field. Referring to FIG. 1B, in an interlace scan display, such as a television monitor, each frame is displayed in two fields. Each field consists of half the frame's lines and the lines of respective fields are interlaced. For example, the "odd" field consists of the odd lines L1, L3, L5, etc., and the "even" field consists of the even lines L2, L4, L6, etc. Interlace scan data is typical of NTSC, PAL and other television standards.
Presently, there are a number of products that employ a television-type (i.e., interlace scan) monitor to display progressive scan computer data. Examples include WebTV, which uses a television monitor to display Internet images, and various network computers and low cost computer systems that allow users to employ a television monitor as a computer display. This class of products is now described in reference to FIG. 2.
Referring to FIG. 2, there is shown a block diagram generally representative of prior art computer systems 140 that display progressive scan data 121 on an interlaced video display 190. Such a computer system 140 includes a processing unit 150, a memory 160 and a scan converter 180. The memory 160, which could be a fast random access memory or a slower hard disk drive, holds an operating system 162, application programs 164 and data 170. In the conventional manner, the processing unit 150 executes application programs 164 in the memory 160 under control of the operating system 162. Non-interlaced data 121 is stored temporarily in the memory 160 as video data 172, which is converted to the interlaced data 141 by the scan converter 180. Filter coefficients 174 used in the various filters employed in the scan converter 180 can also be stored in the memory 160.
The scan converter 180, which can be implemented in hardware or software, generates the interlaced stream 141 in accordance with the vertical and horizontal resolutions and scan rate of the interlaced video display 190. Typically, a scan converter 180 includes a flicker filter 182 to ameliorate image flicker, which is a problem that arises when images are converted from progressive scan to interlace scan. A scan converter 180 can also include a line memory 184 that provides fast, limited storage for the image data being processed by the scan rate converter 180 and the flicker filter 182. The problem solved by the flicker filter 182 is now described in reference to FIG. 3.
Referring to FIG. 3, there is shown one frame 122 of an original image represented by the non-interlaced stream 121, a closeup view 124a of that image, and representations 126a, 128a of the closeup 124a as it would appear on subsequent even and odd fields shown on the interlaced video display 190. Many of the details and horizontal edges of the original image 122 (e.g., the top and bottom lines of the "E" in the closeup 124a) lie along a single horizontal line. In interlaced video, such single-line details are scanned at only half the frame rate, causing annoying flickering on the display 190. For example, the top line of the "E" disappears in the odd field 126a and the bottom line disappears in the even field 128a. Image flicker results when the fields 126a, 128a are displayed sequentially, as they would be on the interlaced display 190.
In the prior art, single dimensional vertical filters (such as the [1, 2, 1] filter 182a of FIG. 2) are used to remove this flicker by blurring the offending detail over three frame lines instead of one. When properly executed, such filters completely remove the observable flickering, but at the cost of greatly reduced vertical image resolution.
Another class of filters tries to take advantage of the fact that, in general, flickering is only noticeable on larger elements of the image. For example, thin horizontal lines on the borders of large block characters or graphics flicker a great deal, but the cross element on a `t`, or the dot on an `i` doesn't seem to flicker at all. To take advantage of this effect, adaptive filters 182b (FIG. 2) have been designed that are enabled only when large elements exist on a single line and are otherwise disabled. Such adaptive filters have not been successful as they must switch between the two modes (enabled and disabled) at the pixel rate, resulting in worse artifacts than the flicker they were designed to reduce. Some types of prior art flicker filters are now described in greater detail.
The fundamental problem that causes flicker is the aliasing of an image along the vertical axis. Aliasing is caused when the sample rate of a data system is insufficient to represent the detail in the data. As stated by Nyquist: a wide sense stationary signal can be completely reproduced without error from a set of samples taken from that signal provided that the signal is strictly band-limited to a known frequency (say, FBW), and the spacing of the samples is uniform at a frequency of at least 2.times. FBW.
In a typical case of converting progressive scan images to interlace scan images, the original image 122 (FIG. 3) comprises 480 lines of video. In interlace format, this image is separated into two distinct fields 126a, 128a of 240 lines each (FIG. 3). Assuming that 480 vertical samples are sufficient to meet the Nyquist criteria for the source image, each field of only 240 samples would contain an alias error. If the two fields 126a, 128a were played back simultaneously, the errors would cancel but, in the interlace format, the fields are played back sequentially. Thus, a distortion in alternate fields results that is seen as flicker.
From a Nyquist perspective, the aliasing errors all exist in the upper half of the vertical frequency spectrum (i.e., aliasing errors occur only for high frequency details, which are details that can only be accurately represented with more than 240 samples). Thus, the prior art includes a class of vertical filters (generally known as half band filters) that are applied to the original picture to remove the upper half of the frequency spectrum and thereby filter out the offending elements and eliminate the flicker. Filters of this class that approximate the Nyquist requirement are very complex often having over 35 taps and large arrays of multipliers. Each tap of such a filter requires a full video line of storage to provide the requisite delay. Another problem is that the application of a Nyquist filter (sometimes called a "Brick Wall" filter) creates some serious distortions of its own, such as ringing and loss of detail. For example, small picture details and sharp edges will disappear along with the flicker. In view of these problems with Nyquist filters, another class of filters has been developed that implement a less stringent "Constant Luminance Principle."
In contrast to the Nyquist criteria, which is based on a mathematical ideal, the constant luminance principle (CLP) rests on perceptual effects. The CLP states: a sampled video signal will have the same overall luminance or intensity level as the original signal provided the original signal has zero content at the sample frequency, where "Overall Luminance" refers to the DC content of the signal. Thus, the CLP says that the DC content (i.e., the brightness or luminance in the signal) of the video signal remains the same so long as there is no content at the sampling rate prior to sampling. Variations in brightness are perceived as flicker. Thus, if an image is processed in accordance with CLP, an isolated detail in a picture will be reproduced without flicker (and at the same brightness as in the original image), even though it may be distorted in shape or slightly blurred.
As mentioned above, flicker is perceived in progressive scan images converted to interlace scan because the apparent intensity of a detail changes between the two fields of a frame. The class of filters designed in accordance with the CLP eliminate these intensity changes by applying a constant luminance filter to the original image data 121 (FIG. 1) before it is processed by the computer system 140. CLP filters don't guarantee that the shape of the detail is the same in both fields, but they do guarantee that the intensity is the same.
The simplest constant luminance filter is the [1, 1], simple, two point, moving average filter. It is rarely used because it is asymmetrical, and creates images with constant luminance, but with motion between fields on single line details. For best results the filter should be symmetrical. This is easily accomplished by convolving two [1, 1] filters to produce a symmetrical result, the well-known [1, 2, 1] filter, which is the simplest, symmetrical constant luminance FIR filter. Application of this filter to an image prior to scan conversion completely eliminates flicker at the cost of blurring of fine details.
Referring to FIG. 4, there is shown one frame 122 of an original image as represented by the non-interlaced stream 121, a closeup view 124b of that image after [1, 2, 1] filtering, and representations 126b, 128b of the closeup 124b as it would appear in subsequent fields on the interlaced video display 190. The [1, 2, 1] filter 182a generates each line F[i] of the filtered data 124b from the non-interlaced data (N1) 121 in accordance with the following equation: EQU F[i]=1/4(NI[i-1]+2NI[i]+NI[i]).
As a result, the top line of the "E", which is on an odd line, is represented on the even field 126b as symmetrical shadows of the original detail. Similarly, the bottom line of the "E", which is on an even line, is represented on the odd field 128b as symmetrical shadows of the original detail. Designers have also tried to modify the [1, 2, 1] filter to reduce its vertical blurring. Typical modifications include [1, 3, 1] and [2, 5, 2] filters. The improvements these filters introduce are incremental and are somewhat offset by a concomitant increase in flicker.
One of the primary concerns in scan conversion is the preservation of text readability in standard television images converted from computer display images. Since the traditional flicker filter necessarily blurs the image, text readability suffers. In an attempt to counter this effect,.some manufacturers have used edge enhancing filters to make the text more readable. This attempt has not been successful largely because the filters operate in the horizontal direction, while the blurring distortion is introduced by flicker filters in the vertical direction. Moreover, adding an edge enhancing filter in the vertical direction simply undoes the action of the flicker filter.
Therefore, it is an object of the present invention to provide a scan conversion system that performs flicker filtering while enhancing the readability of text displayed on an interlaced display.